Letter to the Editor Under Neutrality , Q ST # F ST When There Is Dominance in an Island Model

TO test whether quantitative traits are under directional or homogenizing selection, it is common practice to compare population differentiation estimates at molecular markers (FST) (Wright 1951) and quantitative traits (QST) (Spitze 1993). If the trait is neutral and its determinism additive, then theory predicts that QST 1⁄4 FST, while QST . FST is predicted under directional selection for different local optima, and QST , FST is predicted under homogenizing selection (Merila and Crnokrak 2001). Goudet and Büchi (2006) recently evaluated the effects of dominance, inbreeding, and sampling design on QST for neutral traits. Under dominance, Goudet and Büchi (2006) found that (1) dominance decreases on average the value of QST relative to FST (i.e., QST FST # 0), (2) the magnitude of the contrast QST FST increases with population differentiation (i.e., with increasing FST), and (3) dominance is unlikely to lead to QST FST . 0. In a recent letter to Genetics, Lopez-Fanjul et al. (2007) questioned the evidence leading to these claims. In particular, they criticized Goudet and Büchi (2006) for using averages over allele frequencies and dominance deviations. Here, taking an analytical approach similar to that used in Lopez-Fanjul et al. (2007), we first show that under an island model, the result QST # FST with dominance obtained by Goudet and Büchi (2006) is strictly true over all allelic frequencies and dominance deviations. We then argue that independently of the underlying population structure, averaging over allele frequencies and dominance deviations is pertinent, since quantitative traits are polygenic, and this is what empiricists study when they estimate QST. We conclude by emphasizing that the major problem faced by empiricists is not the slight negative bias in QST due to nonadditive effects, but the very large variance in this quantity, particularly when the number of samples is small. As Lopez-Fanjul et al. (2007) and Goudet and Büchi (2006) used different parameterizations, some clarification might be useful. Goudet and Büchi (2006) used a, d, and a, while Lopez-Fanjul et al. (2007) used 1 s, 1 hs, and 1 for the genotypic values of AA, AB, and BB, respectively. These two notation schemes are equivalent when a 1⁄4 1 (and therefore s 1⁄4 2) and h 1⁄4 ð1 dÞ=2. To obtain the expectation of FST 1⁄4 1 HS=HT and QST 1⁄4 VB=ðVB 1 2VAWÞ, four quantities are needed: gene diversities within [HS 1⁄4 2n 1 p Pnp i qið1 qiÞ] and over all [HT 1⁄4 2 qð1 qÞ] populations, where q represents the frequency of the recessive allele and np the number of populations, as well as the additive variance within (VAW) and between (VB) populations. Under strict additivity, these four quantities are functions of the first and second moments of the distribution of allele frequencies only. However, under dominance (when h 61⁄4 2), VAW and VB also depend on the third and fourth moments of this distribution. As we see below, the difference between Lopez-Fanjul et al. (2007) results and those of Goudet and Büchi (2006) stems partly from the assumed distribution of allele frequencies. We consider exactly the same genetical setup as in Lopez-Fanjul et al. (2007): a biallelic locus with dominance h, additive effect s, and allele frequency q. Using their notation, the mean for the trait, in any given population, is given by M 1⁄4 1 2qhs qs(1 2h). The variance of trait mean among populations is given by VB 1⁄4 E(M ) E(M), where E denotes expectation with respect to the distribution of q among populations. This turns out to be a polynomial function of the first four moments of allelic frequencies. The additive variance in a given population is given by VAW 1⁄4 2aq(1 q) [where a 1⁄4 s(h 1 (1 2h)q is the average effect; Lopez-Fanjul et al. 2003]. With h 61⁄4 2 the expectation of this additive variance among populations, E(VAW), is also a polynomial function of the first four moments of allele frequencies. The expectations of these quantities are obtained by replacing q, q, q, and q in their expressions by the first, second, third, and fourth moments of allele frequency Corresponding author: Department of Ecology and Evolution, Biophore, UNIL-SORGE, University of Lausanne, CH-1015 Lausanne, Switzerland. E-mail: [email protected]